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Abstract 

Students often have differing perceptions of their school’s climate. Although these 
subjective perceptions can be meaningful outcome predictors, discrepancies create challenges for 
those seeking to globally characterize or intervene to improve the climate of a school. Trained, 
outside observers can provide insights on perceptible and ostensibly malleable aspects of the 
school; however, the extent to which these outside observations help us to understand differences 
in students’ report of school climate has not been examined. To study this, we examined 
convergence and divergence between independent observers’ assessments and students’ shared 
perceptions of school climate at the classroom- and school-levels. Data come from the Maryland 
Safe and Supportive Schools Initiative (MDS3), which included 20,647 students and 
observations of 303 teachers in 50 high schools. Students responded to survey items regarding 
safety, engagement, and environment; independent observers assessed teachers’ classroom 
practices and the school physical environment. A three-level model partitioned variance to the 
individual, classroom, and school levels. Most of the variance in students’ climate perceptions 
was between students, with classroom and school intraclass correlation coefficients (ICCs) 
ranging from 0.01 to 0.20; however, observations explained large percentages of between-school 
variation in climate (58% to 91%). Findings suggest the potential utility of outside observations 
for explaining school-level variation in school climate to inform decision-making and future 


research. 
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Triangulating School Climate: 
Areas of Convergence and Divergence Across Multiple Levels and Perspectives 

School climate is a global construct that broadly refers to a school’s physical and 
psychological condition (Thapa, Cohen, Guffey, & Higgins-D’ Alessandro, 2013; Wang & 
Degol, 2016). School climate has been defined as the values, beliefs, and expectations in a 
school that ensure that students feel socially, emotionally, and physically safe (National School 
Climate Council, 2007). The U.S. Department of Education developed a comprehensive model 
for understanding and evaluating school climate, which includes three overarching domains of 
engagement, safety, and environment (Bradshaw, Waasdorp, Debnam, & Lindstrom Johnson, 
2014; U.S. DOE National Center on Safe and Supportive Learning Environments, n.d.). The 
engagement domain includes dimensions reflecting strong and equitable relationships between 
students, teachers, families, and schools and strong connections between schools and the broader 
community; the safety domain assesses the extent to which students feel safe from violence, 
bullying, harassment, and controlled-substance use; and, the environment domain reflects the 
extent to which school facilities are clean and comfortable, classrooms are well-managed, school 
based health supports are available, and there is a clear and fair disciplinary policy. 

These domains of school climate have been assessed by a wide range of indicators. For 
example, the presence of graffiti on the building exterior (Lindstrom Johnson, Bottiani, 
Waasdorp, & Bradshaw, 2018; Wilcox, Augustine, & Clayton, 2006), use of exclusionary 
discipline practices (Bottiani, Bradshaw, & Mendelson, 2017), and students’ feeling of belonging 
and being cared about within the school community (Debnam, Lindstrom Johnson, Waasdorp, & 
Bradshaw, 2014; Payne, 2008) all have been examined as aspects of school climate. School 


climate is traditionally measured by asking students and other school stakeholders about their 


perceptions of myriad aspects of the school, on the premise that a multidimensional, multi- 
informant view will provide a more valid and reliable assessment of climate. Yet students and 
other school community members often differ in their perceptions of the same school’s climate 
(Waasdorp, Pas, O’Brennan, & Bradshaw, 2011), and it may be unclear how to reconcile these 
discrepancies when decisions must be made by administrators affecting the whole school. 

Prior studies indicate that the majority of variability in student’ perceptions of school 
climate are attributable to student-level differences (Fan, Williams, & Corkin, 2011; Koth, 
Bradshaw, & Leaf, 2008). Yet student-level differences in perceptions of climate should not be 
considered idiosyncratic and subjective to the student alone. Systematic gaps in climate 
perceptions are frequently identified based on student social position and identity factors (1.e., 
race, ethnicity, gender, socioeconomic status; Bottiani, Bradshaw, & Mendelson, 2016; Shukla, 
Konold, & Cornell, 2016; Voight, Hanson, O’ Malley, & Adekanye, 2015), suggesting an 
important role of the environment and person-environment interactions. Yet, the extent to which 
the school social and disciplinary environment can explain such gaps is rarely reported (Bottiani 
et al., 2014), and a focus on the school physical environment is even less common (Lindstrom 
Johnson et al., 2018). 

Research that can shed light on the extent to which variability in students’ perceptions of 
school climate may be explained by school and classroom factors is likely to yield insights with 
implications for intervention. Independent observations may help to discern what specific, 
malleable aspects of the school and classroom explain differences in school climate perceptions, 
above and beyond individual-level factors. Toward that end, the current study sought to advance 
our knowledge of these aspects by evaluating the relationship between independently observed 


aspects of school climate at both the classroom- and school-level and student perceptions of 


school climate at the individual-level across the domains of safety, engagement, and 
environment. An enhanced understanding of how observed school and classroom factors relate to 
students’ perceptions of school climate may inform future research and initiatives aimed at 
improving school climate. 
Subjective Measurement of School Climate and Related Outcomes 

The rationale for measuring student perceptions of the school is heavily influenced by the 
bioecological model (Bronfenbrenner & Morris, 2007) and social cognitive theory (Bandura, 
2001). The bioecological model emphasizes the importance of an individual’s unique biological 
make-up, lived experiences, and interactions within a nested context as developmental 
determinants of wellbeing and achievement (Bronfenbrenner & Morris, 2007). Social cognitive 
theory (Bandura, 2001) suggests that students’ perceptions of the school environment influence 
their behavior at school. Thus, student perceptions of school climate are considered important in 
their own right to assess and monitor as an indicator of the effectiveness of initiatives to improve 
climate and youth outcomes (Haynes, Emmons, & Ben-Avie, 1997). Empirical evidence 
supports this premise; students’ subjective perceptions of climate have been associated with 
practically and clinically meaningful student outcomes of academic achievement, mental health, 
and delinquency (Thapa et al., 2013; Wang & Degol, 2016). 
Objective Measurement of School Climate and Related Outcomes 

Having established the importance of measuring students’ subjective perceptions of 
school climate, improving our understanding of factors contributing to shared variance in 
students’ perceptions within classrooms and schools, above and beyond individual-level 
differences, will yield important insights into malleable factors that could be adjusted to improve 


students’ perceptions of climate on a broader-level. Although some research has examined 


school climate perceptions in the aggregate at the school-level (e.g., Jia, Konold, & Cornell, 
2016), there has been less exploration of the utility of outside observers’ assessments of school 
climate at the classroom and school-levels. 

Much of the work on classroom climate is motivated by the literature on teacher 
professional development, evaluation, consultation and feedback to promote more effective 
classroom interactions and teaching practices (Pianta & Hamre, 2009). Aspects of effective 
teaching that have been observationally assessed include proactive behavior expectations, use of 
praise, and opportunities to respond, as well as less effective practices such as reactive and 
punitive disciplinary tactics (Rusby, Crowley, Sprague, & Biglan, 2011). Proactive behavioral 
expectations refers to the setting of clear expectations (Sugai & Horner, 2002). The effective use 
of praise (or approval) to motivate positive learning behaviors is sparing, specific, and 
conditional (Good & Grouws, 1977). The active facilitation of learning through opportunities to 
respond (OTR; Brophy & Good, 1986; Sutherland, Wehby, & Yoder, 2002) reflects a 
combination of instructional talk, prompts, wait time, and feedback. Studies using observational 
methods to assess climate in classrooms demonstrate that effective teaching practices, such as 
setting proactive behavior expectations, use of limited praise, and opportunities to respond, are 
associated with more positive student-reported school climate experiences and higher levels of 
achievement (Allen et al., 2013; Mikami, Gregory, Allen, Pianta, & Lun, 2011). Alternatively, 
reactive and punitive classroom practices are thought to hinder opportunities for trusting teacher- 
student relationships and autonomy necessary for student engagement (Skiba & Losen, 2017). 

Observations of areas outside the classroom environment are less frequently utilized; yet 
recent work suggests they may predict students’ perceptions of school climate (Lindstrom 


Johnson et al., 2018). Specifically, observed negative student behaviors and lack of school 


building illumination were related to student perceptions of disorder and rules and consequences 
which explained student involvement in violence (Lindstrom Johnson et al., 2018). From a 
school-level, observations of the school physical environment have often been grounded in 
theories related to youth engagement in antisocial behavior. Specifically, there has been an 
interest in the application of theories originally developed to characterize neighborhood 
influences, such as social disorganization theory and Crime Prevention Through Environmental 
Design (CPTED; Crowe, 2000), for use in school observations. These theories contend that an 
individual’s risk for involvement in deviant behavior and perception of the environment are 
influenced by community contextual variables. Social disorganization theory and CPTED further 
suggest that observations of the school physical environment can help to capture particular 
school features that may interfere with effective enforcement of positive behavioral norms 
(Plank, Bradshaw, & Young, 2009) and monitoring of student behavior (Astor, Meyer, & Behre, 
1999). As such, observations of the school physical environment have primarily focused on the 
presence of disorder in schools and found associations with teacher-perceived student 
misconduct (Wilcox et al., 2006) and student-reported drug use (Grana et al., 2010). 
Overview of the Current Study 

The present study was motivated by the potential of outside observations of malleable 
classroom social and school physical environment factors to explain, and thus identify points of 
intervention to improve, students’ shared climate perceptions within classrooms and schools 
Toward this end, we leveraged data from multiple sources (i.e., student- and observer-reports) 
and across multiple levels (i.e., individual, classroom, and school) to explore how observations at 


the classroom- and school-level relate to student report of school climate. Findings inform our 


understanding of areas of convergence and the potential value-added associated with different 
perspectives and sources of information on school climate. 
Method 

Procedure 

Data for the current study come from the Maryland Safe and Supportive Schools (MDS3) 
statewide initiative. Recruitment in this project was led by the Maryland State Department of 
Education. After attending informal meetings that provided an overview of the project, interested 
principals voluntarily agreed to participate in the project. Schools sent home informational letters 
with opt-out waivers for parents who did not want their child to participate in data collection. A 
youth assent process was used in reference to the students’ completion of voluntary, anonymous 
online self-report surveys; these measures were administered by school staff across 
approximately 25 classrooms in each school following a standardized written procedure. Each 
student entered a classroom-identifying password to access the survey, which allowed the 
research team to subsequently link and hierarchically nest student survey responses within 
classrooms. Students completed the survey in a core subject class (primarily language arts) to 
ensure that students were only surveyed once. See Bradshaw, Waasdorp, Debnam, & Lindstrom 
Johnson (2014) for additional details. In addition to the survey, the physical environment of the 
school, and approximately 25 classrooms per school, were also observed by trained observers. 
Physical environment observations systematically included, on the exterior, the entrance to 
school grounds, perimeter of school grounds, parking lots, playing fields, and building entrance; 
on the interior, entryway, two hallways, two stairwells, and the cafeteria. Language arts classes 
were prioritized for classroom observations following a random sampling of other core 


instructional classes (e.g., math, science, and social studies) that were observed until we met the 


goal of observing 25 teachers/classrooms (see details on the observational data collection below). 
Data were recorded on a mobile handheld tablet using the Pendragon software. Observers were 
recruited from the surrounding area using job recruitment sites, selected by research team staff 
for interviews, and if selected to participate in training, passed criminal background checks. 
ASSIST observers and SAfETy observers were trained separately. Trainings included an 8-hour 
didactic session followed by in-school practice and reliability assessment. Observers were 
trained to reliability (at least 80% with an expert observer) and underwent recalibration midway 
through the data collection period to ensure consistency of data. These data were approved for 
analysis by the investigators’ Institutional Review Board. 
Participants 

Data from 20,647 students in 303 classrooms and 50 high schools across 12 districts (1.e., 
half of all possible districts in the state) that participated in the MDS3 initiative were analyzed. 
Of the student survey participants, males and females were proportionate to one another, and 
6,054 (29.3%) were Black, 962 (4.7%) were Latino, 9,838 (47.7%) were White, and 2,808 
(13.6%) were another race/ethnicity or multiple race/ethnicities. The sample included slightly 
more 9" graders (27%) and slightly fewer 12" graders (19%). Seventy percent of students 
reported receiving mostly As and Bs on their report card. Gender, race/ethnicity data, grade- 
level, and academic achievement survey reports were missing from about 4.8% of the sample. Of 
303 classroom teachers, approximately 73% were female. Classroom teachers assigned to 
majority students of color were 48.5%. See Table | for descriptive data on students and 
classrooms. The average school enrollment size in the 50 schools was 1236.8 students (SD = 


457.6). The average percent of school enrollment comprised of Black students was 34.5% (SD = 


24.6). The average percentage of students receiving free and reduced-price meals (FARMs) was 
38.0% (SD =17.7). Schools average suspension rate was 15.5% (SD = 12.0). 
Measures 

Student report. Students in high schools participating in the MDS3 initiative were 
surveyed using an anonymous self-report measure of school climate, called the Maryland Safe 
and Supportive Schools (MDS3) School Climate Survey (Bradshaw et al., 2014). Consistent with 
the U.S. Department of Education’s three-factor model of school climate, the measure included 
three primary domains of Safety, Engagement, and Environment. Specifically, Safety was 
assessed using three subscales: a) concerns about bullying (i.e., the extent to which bullying is a 
problem and students at the school try to stop it; Cronbach’s o = .65, four items); b) concerns 
about substance use (1.e., asking to what extent drugs, tobacco, and alcohol consumption were a 
problem at that school; 3-item o = .87); and c) physical safety (i.e., feeling safe walking to school 
and at school and the extent to which weapon-carrying and physical fighting occur at school; 4- 
item a = .65). Three subscales were used to assess Engagement, including a) student-student 
connectedness (1.e., perceptions that students at school help, respect, like, and trust one another; 
5-item a = .88); b) whole school connectedness (i.e., a positive feeling about the school, 
including pride and liking school; 4-item o = .84); and c) culture of equity (1.e., fair and inclusive 
treatment by race, gender, and socioeconomic status, and cultural background; 4-item a = .83). 
Finally, Environment was assessed using the following three subscales: a) rules and 
consequences (1.e., clear, proactively defined, and consistently reinforced positive school 
discipline; 5-item a = .74); b) physical comfort and cleanliness (i.e., comfortable temperature, 


bright and pleasant appearance, and clean spaces; 4-item o =.80); and c) disorder (1.e., students 


are disruptive and disobey rules, staff let students misbehave, and the school has signs of 
disorder such as broken windows and vandalism); 5-item a = .58). 

Teacher observations. Independent observers were trained to reliably assess student and 
teacher interactions in the classroom using the Assessing School Settings: Interactions of 
Students and Teachers (ASSIST; Rusby, Taylor, & Milchak, 2001) observational measure. The 
ASSIST records tallies of specific teacher and student behaviors as well as global ratings of 
classroom social processes during a 15-minute observation period. For the purpose of this study, 
tallies of teacher behaviors were analyzed, including proactive behavior expectations, approvals 
(1.e., praise), opportunities to respond, reactive behavior management, and disapprovals (i.e., 
punitive discipline). The occurrence of these teacher behaviors was tallied by observers over the 
course of the 15-minute observation with each behavior only being counted in one category; one 
behavior could not be counted in multiple teacher behavior tallies. Each observer was required to 
reach an average of 80% inter-observer agreement with a master trainer across three classrooms 
prior to observing independently in study schools. The average training inter-observer agreement 
rate was 87% when these data were collected in the spring of 2014. Inter-observer agreement 
rates were examined again during active data collection and were also 87% (see Pas, Cash, 
O'Brennan, Debnam, & Bradshaw, 2015 and Gaias, Lindstrom Johnson, Bottiani, Debnam, & 
Bradshaw, 2019 for further description of the ASSIST training and reliability). 

School observations. At the school level, the School Assessment for Environmental 
Typology (SAfETy; Bradshaw, Milam, Furr-Holden, & Lindstrom Johnson, 2015) was utilized, 
an observational measure of the school physical environment. This instrument measures three 
global aspects of the school environment: disorder, surveillance, and appearance. Each global 


domain was measured in multiple locations interior and exterior to the school building, including 


hallways, stairwells, building entrances, and cafeterias on the interior and parking lots, playing 
fields, and the entrance to the school grounds on the exterior. To measure the three global aspects 
of school environment used in the present study (i.e., disorder, surveillance, and appearance), a 
facet-representative parceling strategy (Little, Cunningham, Shahar, & Widaman, 2002) was 
applied to the SAfeTy indicators. This strategy combined items based on location (i.e., interior, 
exterior) and item content and involved combining items that were (1) likely to share secondary 
characteristics and (2) likely to exhibit non-negligible residual correlations with one another 
based on shared features (e.g., measured in same location of the school). The resulting indicators 
of disorder included interior and exterior graffiti, property damage, litter, substance use 
paraphernalia, cigarette butts, and exterior only negative behavioral expectations. Indicators of 
surveillance included interior and exterior presence of surveillance video cameras, as well as 
negative behavioral expectations on the interior of the building, and exterior indicators of 
ownership. Indicators of appearance included clean and aesthetically maintained interior spaces, 
well-lit building entrances, murals and artwork on the building interior, and evidence of 
landscaping on the exterior of the building. The appropriateness of these groupings were 
examined via exploratory factor analysis (EFA). Confirmatory factor analysis was conducted in 
Mplus (v7.11; Muthén & Muthén, 2012) thereafter to understand the dimensionality of the three 
constructs using the item parcels as factor indicators. Modification indices were used to guide the 
formation of cohesive, theoretically justifiable constructs. In many cases this meant the removal 
of parcels with a standardized factor loading of less than 0.40, as suggested by Thompson (2004) 
and others. Models were fit at the global level using root mean squared error of approximation 
(RMSEA), standardized root mean residuals (SRMR), and comparative fit index (CFI). At the 


local level, p values resulting from Wald tests on the parameter estimates (loadings, factor 


correlations, etc.) were used to assess fit. Global fit results for the final, three-factor model were 
7 (105) = 589.95, p < .001, CFI = 0.807, TLI = 0.770, RMSEA = 0.105 (95% CI = 0.083, 
0.126), SRMR = 0.092. 
Analyses 

The cross-sectional data were nested such that student self-report (V = 20,647) was 
nested within classrooms (J = 303), which were nested within schools (K = 50). School-level 
design effects, were determined to exceed 2.0 for all outcomes (ranged from 16.2 to 79.6), 
indicating a need to account for the hierarchical structure of the data analytically with multilevel 
modeling (Peugh, 2010). A three-level model was fit to the data in Mp/us, such that student-level 
associations were modeled in a WITHIN model, and classroom- and school-level associations 
were modeled in two BETWEEN models. A step-by-step approach was employed to build the 
three-level model in Mplus, adding one variable and one level at a time in order to be sensitive to 
the stability and significance of findings (Raudenbush & Bryk, 2002). Given the organization of 
school climate around three broad domains of safety, engagement, and environment (U.S. 
Department of Education, 2019), three separate models were run for each domain. Consistent 
with prior research highlighting multiple dimensions within this three-domain model of school 
climate (Bradshaw et al., 2014), the safety model included student-reported perceived safety, 
bullying, and substance use dimensions as outcomes. The engagement model included student- 
reported whole-school connectedness, student connectedness, and culture of equity dimensions 
as outcomes. Finally, the environment model included student-reported school physical comfort, 
rules and consequences, and disorder dimensions as outcomes. 

The predictor variables for all three models were the same. At level-1, these included 


student demographics (e.g., gender, race/ethnicity, grade-level, and academic achievement 


(measured by student self-reported grades). At level-2, we included the ASSIST teacher behavior 
tallies, including proactive behavior expectations, reactive behavior management, approvals, 
disapprovals, and opportunities to respond. We included the percent of the classroom 
composition that was students of color and an indicator of whether the classroom teacher was 
observed to be female or male. We also conducted latent partitioning, so that the ASSIST 
indicators could be used as predictors at level-2 and level-3 (Asparouhov & Muthen, 2006). 
Specifically, the ASSIST was treated as a latent covariate and decomposed into two uncorrelated 
latent variables for use as a predictor at the classroom and school levels. At level-3, we also 
added the SAfETy scores for appearance, surveillance, and disorder, in addition to school-level 
demographics including total student enrollment count, percent of the student enrollment that 
was Black, percent of student enrollment receiving one or more out-of-school suspension(s) that 
year, and percent of student enrollment receiving FARMs. All variables were standardized in 
StataIC 14 prior to running models in Mplus; thus, all variables were grand-mean centered and 
the coefficients reported can be interpreted as effect sizes. 

Missingness of the outcome data, which ranged for the nine outcome variables from 6.6% 
to 15.9%, were determined not to be Missing Completely at Random (MCAR); Missingness at 
Random (MAR) was assumed. Full-information maximum likelihood (FIML) estimation with 
robust standard errors in Mplus was used to account for missing data in analyses (Enders & 
Bandalos, 2001). Although all models terminated normally, Mp/us produced a warning that was 
determined through sensitivity analyses to be caused by inclusion of binary covariates in the 
model, a warning that the software developers have specified as ignorable. We conducted further 


sensitivity analyses removing the binary covariates and confirmed all models terminated 


normally with no warnings and we found the significance, size, and directionality of the 
estimates at levels 2 and 3 were substantively unchanged. 
Results 

Model Fit, Partitioning of Variance, and Variance Explained 

Across models, most of the variation in student perceptions of school climate was 
attributable to the student-level (ranging from 76-94% of the unadjusted variance). In contrast, 
within-student R* were low, ranging between 1-5% across models, when including student 
race/ethnicity, gender, grade-level, and self-reported achievement as student-level predictors. 

Safety. As reported in Table 2, the Safety model demonstrated adequate fit, y°(65) = 
134.70, p < .001, TLI = 0.993, CFI = .985, RMSEA = .007. Unadjusted level-3 intraclass 
correlation coefficients (ICCs) ranged from 0.071 to 0.073 for students’ perceptions of physical 
safety, bullying as a problem at school, and substance use as a problem at school. The between- 
school R? in the final model with all covariates and latent partitioning of the ASSIST at level-3 
was 77% for physical safety, 83% for bullying as a problem at school, and 91% for substance use 
as a problem at school. By comparison, unadjusted level-2 ICCs were 0.018 for physical safety, 
0.013 for bullying as a problem at school, and 0.024 for substance use as a problem at school. 
The between-classroom R? in the final model with all covariates and latent partitioning of the 
ASSIST was 3% for physical safety, 7% for bullying as a problem at school, and 15% for 
substance use as a problem at school. 

Engagement. As reported in Table 3, the Engagement model demonstrated adequate fit, 
(65) = 119.75, p < .001, TLI = 0.991, CFI = .996, RMSEA = .007. Unadjusted level-3 ICCs 
were 0.066 for whole school connectedness, 0.076 for student connectedness, and 0.039 for 


culture of equity. The between-school R? in the final model with all covariates and latent 


partitioning of the ASSIST was 74% for whole school connectedness, 90% for student 
connectedness, and 71% for culture of equity. By comparison, unadjusted level-2 ICCs were 
0.013 for whole-school connectedness, 0.012 for student connectedness, and 0.011 for culture of 
equity. The between-classroom R? with all covariates and latent partitioning of the ASSIST was 
10% for whole-school connectedness, 8% for student connectedness, and 6% for equity. 

Environment. As reported in Table 4, the Environment model demonstrated adequate fit, 
7(65) = 136.99, p < .001, TLI = 0.968, CFI = .986, RMSEA = .008. Unadjusted level-3 ICCs 
were 0.201 for physical comfort, 0.051 for rules and consequences, and 0.072 for disorder. The 
between-school R? in the final model with all covariates and latent partitioning of the ASSIST at 
was 59% for physical comfort, 67% for rules and consequences, and 58% for disorder. By 
comparison, unadjusted level-2 ICCs were 0.021 for physical comfort, 0.012 for rules and 
consequences, and 0.007 for disorder. The between-classroom R? in the final model with all 
covariates and latent partitioning of the ASSIST was 11% for physical comfort, 4% for rules and 
consequences, and 7% for disorder. 
Student-Level Associations Across Models 

Student-level associations were found after partitioning out variance attributable to 
classroom and school clustering. Most notably, female students had significantly less favorable 
report of school climate compared to males across all nine dimensions, including physical safety 
(B = -0.04, p < .001), bullying as a problem at school (8 = 0.10, p < .001), substance use as a 
problem at school (8 = 0.10, p < .001), whole-school connectedness (f = -0.09, p < .001), student 
connectedness (B = -0.15, p < .001), culture of equity (B = -0.04, p < .001), physical comfort (B = 
-0.09, p < .001), rules and consequences (B = -0.03, p < .001), and disorder (B = 0.03, p < .001), 


regardless of their school or classroom. Additional significant associations of other individual- 


level predictors (i.e., race, ethnicity, grade-level, and self-reported achievement) are presented in 
Tables 2-4. 
Classroom-Level Associations Across Models 

After partitioning the variance attributable to the individual- and school-levels, a small 
number of significant associations were found at the classroom-level. Specifically, ASSIST 
tallies of teachers’ proactive behavior expectations at level-2 was significantly associated with 
lower levels of student perceptions that bullying and substance use were problems at school 
(both B = -0.02, p < .01 and .05 respectively), as well as higher levels of whole-school and 
student connectedness (B = 0.03, p < .05 and B = 0.02, p < .05, respectively). In addition, OTRs 
were associated with higher levels of student-reported whole-school connectedness and physical 
comfort at school (both B = 0.02, p < .05). No other ASSIST tallies of teachers’ behaviors were 
associated with student perceptions of climate at the classroom level. 
School-Level Associations Across Models 

Portioning the latent variance in the ASSIST observations at the school-level, we found 
that higher observer counts of teachers’ use of approvals in the classroom (i.e., observations of 
teachers giving praise or tangible rewards students) were associated with higher report student 
connectedness at the school-level (6B = 0.30, p < .05), whereas higher counts of teachers’ use of 
disapprovals in the classroom (1.e., observations of teachers making critical remarks or taking 
punitive actions) were associated with lower report of whole-school connectedness (B = -0.18, p 
< .001), student connectedness ($8 = -0.15, p < .05), culture of equity (B = -0.11, p < .031), and 
rules and consequences (f = -0.13, p < .01). We also found that higher observer counts of 
teachers giving students opportunities to respond in the classroom (at the school-level) were 


associated with higher student reports of physical comfort at school (f = 0.02, p < .05) and 


significantly lower student reports that substance use was a problem at school (B = 0.26, p < .05). 
No other associations for the ASSIST were significant at the school-level. 

Observers’ ratings of positive school appearance, as measured by the SAfETy, were 
significantly associated with higher student report that substance use was a problem at school (8 
= 0.10, p < .001); however, it was also associated with higher student report of physical comfort 
at school (6 = 0.19, p < .01). Observers’ ratings of school disorder, as measured by the SAfETy, 
were significantly associated with less favorable student report of climate across five of nine 
dimensions, including less student-perceived physical safety (B = -0.08, p < .05), less whole- 
school connectedness (B = -0.09, p < .05), less student connectedness (f = -0.08, p < .05), more 
bullying as a problem at school (6 = 0.08, p < .05), more substance use as a problem at school (f 
= 0.15, p < .001), and more student-perceived school disorder (f = 0.10, p < .05). Observers’ 
ratings of school surveillance, as measured by the SAfETy, was also significantly associated 
with lower student reports that substance use was a problem at their school (B = -0.05, p < .05), 
but also lower student reports of whole-school connectedness (f = -0.06, p < .05), student 
connectedness (B = -0.04, p < .05), and positive disciplinary environment (rules and 
consequences; B = -0.08, p < .01). Significant associations of school-level covariates including 
enrollment size, percent Black enrollment, suspension, and FARMs are presented in Tables 2-4. 

Discussion 

This study examined the associations of observed, malleable classroom teacher practices 
and school physical environmental factors with students’ perceptions of safety, engagement, and 
environment using a multi-level modeling analytic framework. We sought to identify discrete 
factors that could be modified to improve students’ perceptions of school climate. 


Student-Level Variance in Student-Reported Climate 


Perhaps not surprisingly, this study’s multilevel models demonstrated that variance in 
students’ report of school climate attributable to the student-level ranged between 76-96%. These 
findings align with previous literature indicating that the majority of variability in student 
perceptions of school climate is nested at the individual level (Fan et al., 2011; Koth et al., 2008). 
It is noteworthy that student binary gender, race, grade-level, and self-reported academic 
achievement explained very little of the variance in climate perceptions at the student-level (not 
more than 5%). This finding suggests additional research is needed to explain more of the 
variance at the student-level in perceived climate — particularly because variance at the student- 
level represents the largest share of the variation overall. Unfortunately, it is not common 
practice in the field to present the within R? (individual-level variance explained) when 
examining factors predicting student perceptions of school climate. Better understanding of these 
individual-level factors, and whether they are discrete and modifiable, could inform intervention 
development and decision-making to promote positive climate. 

Despite only explaining up to 5% of the level-1 variance, student identity and social 
position indicators (e.g., gender, race) were associated with significant differences in perceived 
climate, consistent with prior school climate research in the community psychology literature 
(e.g., Shukla et al., 2016; Voight et al., 2015). Gender disparities were most salient. The 
consistent pattern of less favorable climate perceptions for girls, across all outcomes measured in 
the study, suggests the need for intervention focused on improving factors in the school 
environment that may be contributing to gender disparities in climate experiences. We also found 
that when a broader range of indicators of school climate are included, and when observed levels 
of classroom and school climate are modeled as predictors, the racial school climate gap looks 


different than highlighted in prior research (Voight et al., 2015; Bottiani et al., 2016). 


Specifically, although Black students reported less connectedness with their peers (consistent 
with the focus of prior research on racial disparities in school climate), they reported more 
favorable aspects of school climate in other areas. For example, on average, Black students 
reported lower perceptions of school disorder, higher levels of their school being comfortable 
and clean, lower perceptions that substance use was a problem at their school, and lower 
perceptions that bullying was a problem at their school. These findings controlled for classroom 
teacher practices and observer-assessed school disorder, appearance, and surveillance. Our 
findings suggest that when in similar classroom and school settings, as measured by these 
observational tools, Black students may perceive some aspects of school climate more favorably 
than White students. 
Classroom- and School-Level Variance in Perceived Climate 

The extent to which variance was nested within classrooms and schools varied across 
school climate outcomes and levels in this study (see ICC summaries at the bottom of Tables 3- 
5). In one striking instance, a very large percent of the variance was nested at the school-level 
(20.1%) in students’ report that their school was physically comfortable and clean. Specifically, 
this scale asked students to rate the extent to which the temperature was comfortable, the school 
had a bright and pleasant appearance, and spaces (like bathrooms) were clean. Ostensibly, this 
relatively high ICC may be because these physical aspects of the school are less personally 
experienced than features of the social environment asked about in the other subscales. Other 
subscales’ ICCs at the school level ranged between 4-8%. As might be expected, relative to the 
school level, less variance was attributable to the classroom level (.7% to 2.4%). A lower 
proportion of variance in student report of school climate attributable to the classroom was 


expected because of the high school setting, in which students are frequently changing 


classrooms throughout the day and experiences in a single classroom setting are unlikely to drive 
their experience of the school climate more broadly. The setting level findings of this study are 
noteworthy, in particular because we were able to predict large amounts of variance at the 
classroom and school levels with the included class and school predictors. Across three models 
of safety, engagement, and environment, observed and contextual factors explained between 3- 
15% of the shared variance in students’ perceptions of climate at the classroom-level and 
between 58-91% of the shared variance in students’ perceptions of climate at the school-level. 
Notably, the models accounted for 80% or more of the variance in school-level perceived 
bullying and substance use as problems at school, and student connectedness. Additionally, it is 
noteworthy that shared variance in ASSIST classroom observations of teacher practices at the 
school-level was a key determinant of students’ perceptions of school climate. This may have 
been particularly true as this study was of high schools, where students transition to multiple 
teachers and classrooms in a day, as compared to elementary and middle school settings, where 
exposure across multiple teachers and classrooms is more limited. Therefore, studies of 
secondary school contexts should consider how an aggregated experience of teacher behaviors 
and classroom experiences contributes to the overall experience of a school. 
Classroom- and School-Level Associations 

A number of significant classroom-level and school-level associations were found in this 
study. Adjusting for student and school characteristics (e.g., race/ethnicity and FARMs), we 
found that an increase in observed classroom teacher proactive behavior expectations was 
significantly associated at the classroom-level with less student-reported bullying and substance 
use as problems at school, and more student-reported sense of connectedness to their school and 


their peers. Conversely, increases in observed teacher disapproving statements and tangible 


punitive consequences directed at students in the classroom was significantly associated, at the 
school-level, with lower levels of positive school climate, including whole-school 
connectedness, student connectedness with their peers, perceived school culture of equity, and 
disciplinary environment (rules and consequences). Similarly, approvals at the school-level were 
associated with more favorable student connectedness with their peers. Indicators of disorder in 
the physical environment, such as graffiti, were significantly associated with higher student- 
reports of bullying as a problem at school, substance use as a problem at school, and disorder, 
and lower student-reports of perceived safety and whole-school and student connectedness. 

In light of national initiatives on school safety emphasizing target hardening approaches 
(i.e., use of surveillance cameras, metal detectors, and deployment of school police officers), the 
study findings on observed surveillance are striking. Whereas observed surveillance at the 
school-level was associated with lower perceptions that substance use was a problem at school, 
suggesting the possible benefits of this approach, observed surveillance was also associated with 
lower student self-report of feeling connected to their peers and to their school as a whole. 
Students also perceived the school disciplinary environment (rules and consequences) less 
favorably in schools with higher levels of observed surveillance tactics. A sense of 
connectedness to school and experience of positive discipline practices at school have been 
linked longitudinally to a wide range of positive student outcomes, including critical academic, 
social-emotional, and behavioral indicators (Bond et al., 2007; Bradshaw, Mitchell, & Leaf, 
2010; Bradshaw, Zmuda, Kellam, & Ialongo, 2009). These findings highlight the importance of 
weighing the possible benefits of introducing surveillance measures in schools alongside the 
potential costs to students’ social-emotional development and academic success at school 


(Lindstrom Johnson, Bottiani, Waasdorp, & Bradshaw, 2018). 


Observed positive school appearance was associated with higher student perceptions that 
their school was comfortable and clean, but also with higher student perceptions that substance 
use was a problem at their school. To explain the latter finding, consider that better observer- 
rated appearance indicators included, for example, well-landscaped grounds; thus, positive 
appearance may also be an indicator that the school is located in a relatively wealthier 
neighborhood and draws students from families that are higher SES. Students in high 
socioeconomic groups have been shown to be at higher risk of using certain substances (e.g., 
heavy alcohol consumption; Botticello, 2009). Establishing linkages between the socioeconomic 
status of the student enrollment and positive appearance of the school will be important to better 
understand the association between appearance and higher report that substance use is a problem 
at school. In addition, future research could seek to ascertain which substances students consider 
to be a problem at school (1.e., binge drinking, use of illicit substances like heroin or cocaine, 
misuse of prescription opioids like Vicodin or stimulants like Adderall, or regular tobacco 
cigarettes or e-cigarettes and nicotine vaporizers like Juul). 

Together with school administrative data on overall school characteristics (e.g., FARMs), 
these outside observations explained a substantial amount of the variance in students’ 
perceptions of school climate between schools and, to lesser extent, between classrooms. These 
findings also inform our understanding of theories undergirding school climate research as well 
as guiding assessment and intervention efforts. For example, the association of classroom 
observations of teachers’ behaviors in relation to students’ perceptions of climate suggests a 
substantive role of teacher behaviors in the classroom in the school’s broader climate (Allen et 
al., 2013; Mikami et al., 2011; Pianta et al., 2008). This seemed to be particularly true of 


observation of proactive behavioral management and approvals, which are integral to some 


whole-school behavior management preventive interventions (e.g., Positive Behavioral 
Interventions and Supports [PBIS]), in which consistently setting expectations and rewarding 
positive behaviors is a critical component (Sugai & Horner, 2002). Relatedly, disapprovals 
(critical and punitive teacher classroom practices) were associated with less connectedness and 
equity, two dimensions of climate that prior research has identified as disparities affecting Black 
students and potentially related to disproportionate discipline practices. This finding is consistent 
with prior research advocating a shift away from punitive and critical discipline to reduce 
disproportionality (Skiba & Losen, 2017). 

Finally, the findings highlighted the important role of the physical environment in 
determining student perceptions of climate. Consistent with theory (CPTED; Crowe, 2000), 
observed school disorder was negatively related to student connectedness with their peers and 
perceived physical safety (Grana et al., 2010; Wilcox et al., 2006). While the study of the school 
physical environment has primarily been supported by theories of social control, these findings 
support the idea that introducing monitoring features to the school physical environment can 
serve as not only a constraint on negative social interactions (e.g., surveillance to minimize illicit 
behavior), but also to positive and prosocial behaviors, too. 

Limitations and Future Directions 

Despite a number of strengths (e.g., sample size, multiple sources of data) of this study, 
some aspects of the study are constraining, and these are important to note. Specifically, the 
measures of climate at each level were related to one another, but not designed to measure 
precisely the same constructs. This is in part because observers could not be expected to perceive 
aspects of the school that are personally experienced (e.g., whether or not students feel connected 


to peers at school). Thus, constructs assessed across informants were conceptually related, but 


did not in every instance directly overlap. Nevertheless, the alignment across several of the 
various dimensions of the broader school climate constructs speaks to the robustness of the 
findings (e.g., school-level disorder as measured by the SAfETy was associated positively and 
significantly with student-reported disorder at school). Future research is needed to further 
inform our understanding of more precisely aligned indicators of school climate. 

Two other measurement considerations should be noted. One is that classroom 
observations were limited to 15 minutes due to resources required to observe 303 classrooms in 
50 schools; although this may seem to be a small window into the classroom and school, our 
sampling strategy of 25 classrooms in core subjects allows us to make broader inferences about 
students’ school experiences from these observations. Second, a few of the outcome scales for 
this study had relatively low internal consistency reliability (i.e., .58-.74); however, the majority 
of the scales had very good internal consistency reliability, ranging from .80 to .88. 

We were also somewhat limited to a relatively narrow set of self-reported student 
characteristics, including academic achievement, race, and gender; a broader set of 
characteristics may provide a more complete understanding of the extent to which perceptions 
vary as a function of student characteristics. Similarly, we explored school-level demographic 
characteristics that were available through archival data. We also relied on external observers’ 
report of the percentage of students of color present within the classrooms. Inclusion of 
additional information on classrooms and teachers could be further informed by teacher self- 
report data on their own experiences of the school environment, which in turn may have 
influenced their behaviors in the classroom and use of different behavior management and 
student engagement strategies. It may also be helpful to model teachers’ own perspectives of 


climate as yet another perspective on the classroom and school environments in relation to the 


students’ and external observers’ perceptions. Additionally, future research should explore the 
relationship between observed climate and student academic, health and mental health, and 
behavioral outcomes (e.g., academic achievement, social emotional learning indicators, 
graduation, contact with juvenile justice system). 
Conclusion 

Taken together, these findings enhance our understanding of areas of convergence and 
divergence of independent observations of climate in relation to students’ viewpoints in a 
multilevel context. They suggest the importance of understanding variability in students’ 
perceptions of school climate as explainable, both in terms of individual-factors as well as 
classroom and school-level conditions. Implications include the future integration of 
observational measures in comprehensive school climate assessment, including the potential 
value of a multi-informant approach to school climate. This could involve expanding already 
existing observational tools such as classroom walkthrough and school safety audits to assess 
aspects of school climate more broadly. Such work may serve to inform school improvement 
efforts aimed at improving the various aspects of school climate examined in the current study. 
In this way schools may have more explicit ideas about how to improve school climate, a 


construct with known implications for student well-being and academic success. 
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Table 1 
Demographic Characteristics of Students and Classrooms 


Student Characteristics (NV = 20,647) N (%) 
Gender 
Male 9,751 (47.2) 
Female 9,914 (48.0) 
Missing Gender Data 982 (4.76) 
Race/Ethnicity 
Black 6,054 (29.3) 
Latino 962 (4.7) 
White 9,838 (47.7) 
All Other (Multiracial, American Indian, and Asian/Pacific Islander) 2,808 (13.6) 
Missing Racial/Ethnic Data 985 (4.8) 
Grade 
Grade 9 5,584 (27.05) 
Grade 10 5,278 (25.6) 
Grade 11 4,848 (23.5) 
Grade 12 3,281 (19.2) 
Missing Grade Data 971 (4.7) 
Academic Achievement (Self-Reported) 
Mostly As 6,688 (32.4) 
Mostly Bs 7,765 (37.6) 
Mostly Cs 3,978 (19.3) 
Mostly Ds 868 (5.9) 
Mostly Fs 356 (1.7) 
Missing Achievement Data 992 (4.8) 
Classroom Characteristics (J = 303) N (%)/M (SD) 
Female Teacher Gender 221 (72.9) 
Classroom Composition is Majority Students of Color 147 (48.5) 
School Characteristics (K = 50) N (%)/M (SD) 
Enrollment 1236.8 (457.6) 
Percent Black 34.5 (24.6) 
Percent Suspended 15.50 (12.0) 
Percent FARMs 38.03 (17.7) 
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Table 2 
Associations of the Three-Level Model of Safety 
Perceived School Safety Bullying as Problem at School Substance Use as Problem at School 
(Student-Reported) (Student-Reported) (Student-Reported) 

Student Level (N = 20,647) y SE t Pp y SE t 7) y SE t 7) 
Female -0.04 0.01 -3.91 .000 0.10 0.01 12.88  .000 0.07 0.01 7.11 .000 
Black (ref. White) 0.01 0.01 0.90 .370 -0.09 0.01 -8.07 .000 -0.12 0.01 -9.57  .000 
Latino (ref. White) -0.01 0.01 -1.05 .293 -0.02 0.01 -1.65  .100 -0.01 0.01 -1.50  .135 
Other (ref. White) -0.06 0.01 -6.15 .000 0.00 0.01 -0.42  .677 -0.01 0.01 -0.94  .345 
Grade-Level 0.02 0.02 0.95 .342 -0.06 0.01 -4.12  .000 0.06 0.02 4.03 .000 
Academic Achievement 0.14 0.02 9.16 .000 -0.05 0.01 -3.81 000 0.01 0.01 0.94 _.350 
Classroom Level (J = 303) y SE t 7) y SE t 7) y SE t 7) 
Proactives (ASSIST) 0.02 0.01 1.29  .197 -0.02 0.01 -2.60 .009 -0.02 0.01 -2.45 .014 
Approvals (ASSIST) 0.01 0.01 0.86 .388 0.00 0.01 -0.53 598 0.01 0.01 0.87 .385 
Disapprovals (ASSIST) 0.00 0.01 -0.32  .748 -0.01 0.01 -0.48  .634 -0.01 0.01 -0.36  .717 
Reactives (ASSIST) 0.00 0.01 -0.08 — .935 0.00 0.01 -0.29  .776 -0.02 0.01 -1.58 115 
OTRs (ASSIST) 0.00 0.01 0.33. .742 0.00 0.01 -0.16  .873 0.00 0.01 0.33  .740 
% Students of Color (ASSIST) 0.01 0.02 0.45 .652 0.01 0.02 0.30 .766 -0.04 0.02 -1.75 079 
Female Teacher (ASSIST) 0.01 0.01 0.71 .476 0.01 0.01 0.85 398 0.00 0.01 O37. 3715 
School Level (K = 50) y SE t Pp y SE t Pp y SE t Pp 
Proactives (ASSIST) 0.05 0.16 0.28 .778 -0.29 0.21 -1.43 .153 0.07 0.10 0.75 452 
Approvals (ASSIST) 0.29 0.21 141 .159 -0.24 = =0.22 -1.06 .288 -0.24 0.20 -1.20— .231 
Disapprovals (ASSIST) -0.09 0.06 -1.62 .106 0.11 0.07 164 102 0.02 0.05 0.40 689 
Reactives (ASSIST) -0.06 0.06 -0.94 349 0.06 0.07 0.96  .337 0.02 0.06 0.40 .690 
OTRs (ASSIST) -0.17 0.18 -0.97.  .330 0.25 0.19 1.32 .188 0.26 0.10 2.51 .012 
Appearance (SAfETy) 0.00 0.03 0.06 .954 -0.01 0.04 -0.13 895 0.10 0.03 3.74  .000 
Disorder (SAfETy) -0.08 0.04 -2.35 .019 0.08 0.04 2.28  .022 0.15 0.03 4.99  .000 
Surveillance (SAfETy) -0.03 0.03 -1.22 221 0.01 0.03 0.42 673 -0.05 0.02 -2.35 .019 
Enrollment Size -0.03 0.04 -0.83 .408 0.06 0.04 Loe - lb 0.09 0.03 2.80  .005 
% Black -0.02 0.04 -0.46 —.647 -0.02 0.04 -0.48 628 -0.18 0.03 -6.20 .000 
% Suspended -0.03 0.03 -1.31  .189 0.10 0.03 3.36 =. .001 0.01 0.02 O95). 408d 
% Free & Reduced-Price Meals -0.17 0.03 -4.79  .000 0.16 0.04 4.47 .000 0.16 0.02 6.66 .000 
Classroom ICC 0.02 0.01 0.02 

School ICC 0.06 0.08 0.05 

R? - Within 0.03 0.02 0.02 

R? — Between Class 0.03 0.07 0.15 


R? — Between School 0.77 0.83 0.91 
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Note. Model Fit: y? (65) = 134.70, p <.001; RMSEA = .007; TLI = .99, CFI = .99. Proactives = Proactive behavior expectations, Reactives = Reactive behavior management, 
Approvals = Approval and tangible reward; Disapprovals = Disapproval and punitive consequence; OTRs = Opportunities to respond. ICC= Intra-class correlation coefficient. 
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Table 3 


Associations of the Three-Level Model of Engagement 
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Whole School Connectedness 
(Student-Reported) 


Student Connectedness 
(Student-Reported) 


Culture of Equity 
(Student-Reported) 


Student Level (NV = 19,621) y SE t Pp y SE t 7) y SE t 7) 
Female -0.09 0.01 -12.16 .000 -0.15 0.01 -19.99 .000 -0.04 0.01 -3.98  .000 
Black (ref. White) -0.01 0.02 -0.80  .425 -0.05 0.02 -3.34 .001 -0.02 0.02 -0.78  .436 
Latino (ref. White) 0.01 0.01 1.33 .185 -0.01 0.01 -1.07  .286 -0.02 0.01 -2.65 .008 
Other (ref. White) -0.02 0.01 -1.81  .071 -0.04 0.01 -3.19 .001 -0.06 0.01 5.27 —.000 
Grade-Level -0.03 0.02 -1.88  .060 -0.02 0.01 -1.45  .146 -0.02 0.01 -1.42  .155 
Academic Achievement 0.21 0.01 20.95  .000 0.15 0.01 12.68  .000 0.11 0.01 11.50  .000 
Classroom Level (J = 303) y SE t 2) y SE t 7) y SE t 7) 
Proactives (ASSIST) 0.03 0.01 2.26 ~=.024 0.02 0.01 2.11 = .035 0.02 0.01 1.91 .056 
Approvals (ASSIST) -0.01 0.01 -0.64 .520 0.00 0.01 0.23 .820 -0.01 0.01 -0.67 .506 
Disapprovals (ASSIST) 0.01 0.02 0.37 .714 -0.01 0.01 -0.98  .327 -0.01 0.01 -0.84  .403 
Reactives (ASSIST) 0.00 0.01 0.15  .879 0.00 0.01 -0.11 .910 0.00 0.01 0.48 .628 
OTRs (ASSIST) 0.02 0.01 2.18 .029 0.01 0.01 1.43 .153 0.00 0.01 -0.29 .772 
% Students of Color (ASSIST) -0.01 0.02 -0.44 .660 -0.01 0.02 -0.55 .582 -0.01 0.02 -0.47 637 
Female Teacher (ASSIST) 0.01 0.01 1.37 .171 0.01 0.01 1.10  .271 -0.01 0.01 -0.90  .367 
School Level (K = 50) y SE t Pp y SE t y SE t Pp 
Proactives (ASSIST) 0.18 0.19 0.98  .329 0.25 0.16 1.54 .124 0.14 0.12 1.19 .236 
Approvals (ASSIST) 0.25 0.17 1.52  .128 0.30 0.14 2.20 = .028 0.14 0.17 0.82 415 
Disapprovals (ASSIST) -0.18 0.05 -3.65 .000 -0.15 0.06 -2.57  .010 -0.11 0.05 -2.15 .031 
Reactives (ASSIST) -0.08 0.07 -1.25  .210 -0.10 0.06 -1.76 .079 -0.07 0.06 -1.12  .261 
OTRs (ASSIST) -0.17. 0.18 -0.90 .370 -0.25 0.19 -1.27  .204 -0.32 0.18 -1.85  .064 
Appearance (SAfETy) 0.03 0.04 0.79 428 0.01 0.03 0.27 ~—.785 0.04 0.03 1.48 .139 
Disorder (SAfETy) -0.09 0.04 -2.26 .024 -0.08 0.04 -2.26 .024 -0.04 0.03 -1.12 .262 
Surveillance (SAfETy) -0.06 0.03 -2.18 .029 -0.04 0.02 -2.18 .029 -0.03 0.02 -1.23 .218 
Enrollment Size 0.03 0.03 0.84 .402 0.01 0.03 0.19 .848 0.02 0.03 0.65 .516 
% Black 0.01 0.04 0.18  .855 -0.01 0.03 -0.21  .833 0.04 0.03 1.44  .150 
% Suspended -0.05 0.03 -1.95 .051 -0.05 0.02 -2.03 .042 -0.07 0.03 -2.76 .006 
% Free & Reduced-Price Meals -0.08 0.04 -1.94  .053 -0.13 0.03 -3.91  .000 -0.04 0.04 -0.96 335 
Classroom ICC 0.02 0.01 0.01 

School ICC 0.05 0.06 0.04 

R? - Within 0.05 0.05 0.02 

R? — Between Class 0.10 0.08 0.06 

R’ — Between School 0.74 0.90 0.71 
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Note. Model Fit: y? (65) = 119.75, p <.001; RMSEA = .007; TLI = 1.00, CFI = .99. Proactives = Proactive behavior expectations; Reactives = Reactive behavior management 
Approvals = Approval and tangible reward; Disapprovals = Disapproval and punitive consequence; OTRs = Opportunities to respond. ICC= Intra-class correlation coefficient. 


Table 4 
Associations of the Three-Level Model of Environment 
Physical Comfort Rules and Consequences School Disorder 
(Student-Reported) (Student-Reported) (Student-Reported) 

Student Level (NV = 19,621) y SE t 2) y SE t 7) y SE t 7) 
Female -0.09 0.01 -7.04  .000 -0.03 0.01 -3.49 000 0.03 0.01 3.15 .002 
Black (ref. White) 0.03 0.01 2.31 =.021 0.02 0.01 1.83 067 -0.10 0.01 -7.94  .000 
Latino (ref. White) 0.01 0.01 1.93 .054 0.02 0.01 1.79 .073 -0.03 0.01 -3.11 .002 
Other (ref. White) 0.01 0.01 2.07 .038 -0.02 0.01 -1.52  .128 -0.01 0.01 -1.21 .226 
Grade-Level -0.02 0.02 -1.15 .252 0.03 0.01 2.30 .021 0.00 0.01 0.12 .908 
Academic Achievement 0.06 0.01 5.17 _.000 0.12 0.01 16.85 .000 0.00 0.01 0.13. .897 
Classroom Level (J = 303) y SE t 2) y SE t Pp y SE t 7) 
Proactives (ASSIST) 0.00 0.01 0.23 .818 0.01 0.01 0.54  .589 0.01 0.01 133° .2eh27 
Approvals (ASSIST) -0.02 0.01 -1.79  .074 0.02 0.01 146 144 0.00 0.01 -0.34 —.736 
Disapprovals (ASSIST) 0.00 0.02 0.02 .988 -0.01 0.02 -0.41 685 0.01 0.02 0.59 557 
Reactives (ASSIST) 0.02 0.01 1.64 102 0.00 0.01 -0.10  .923 -0.01 0.01 -1.05 .292 
OTRs (ASSIST) 0.02 0.01 2.12 .034 0.01 0.01 0.63 .531 -0.01 0.01 -1.31  .190 
% Students of Color (ASSIST) -0.03 0.02 -1.85 .064 -0.01 0.02 -0.42  .677 -0.01 0.02 -0.44 657 
Female Teacher (ASSIST) -0.01 0.01 -1.08  .278 0.00 0.01 0.07 .941 0.01 0.01 07% 3477 
School Level (K = 50) y SE t Pp y SE t Pp y SE t Pp 
Proactives (ASSIST) -0.09 0.24 -0.39 695 0.10 0.12 0.81 417 0.08 0.18 0.43 .671 
Approvals (ASSIST) -0.41 0.30 -1.39 .165 0.18 0.18 EOl, 312 -0.01 0.27 -0.04 .965 
Disapprovals (ASSIST) -0.13 0.08 -1.66  .097 -0.13 0.04 -3.09 .002 0.10 0.07 E37. Ad 
Reactives (ASSIST) 0.00 0.12 0.02 .987 -0.12 0.06 -1.99  .046 0.04 0.08 0.50 .619 
OTRs (ASSIST) 0.06 0.31 0.18  .860 -0.12 0.16 -0.76 447 0.08 0.22 0.36 .717 
Appearance (SAfETy) 0.19 0.06 3.23 = .001 0.01 0.03 0.36 .720 -0.04 0.04 -0.89 373 
Disorder (SAfETy) -0.14 0.08 -1.74  .081 -0.07 0.04 -1.84  .065 0.10 0.05 2.22 .026 
Surveillance (SAfETy) 0.01 0.05 0.19  .850 -0.08 0.02 -3.38 .001 -0.01 0.03 -0.17 869 
Enrollment Size -0.03 0.06 -0.42  .678 0.01 0.03 0.24 810 0.07 0.05 159" 113 
% Black -0.02 0.06 -0.44 663 -0.01 0.03 -0.27. 790 0.03 0.04 0.71  .480 
% Suspended -0.02 0.04 -0.59 554 -0.05 0.02 -2.18  .029 0.05 0.03 1.85  .064 
% Free & Reduced-Price Meals -0.12 0.07 -1.82 .069 -0.08 0.03 -2.48 — .013 0.12 0.04 3.29 001 
Classroom ICC 0.02 0.01 0.01 

School ICC 0.20 0.04 0.08 


R? - Within 0.01 0.02 0.01 
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R? — Between Class 0.11 0.04 0.07 
R? — Between School 0.59 0.67 0.58 


Note. Model Fit: y? (65) = 136.99, p <.001; RMSEA = .008; TLI = .97, CFI = .99. Proactives = Proactive behavior expectations; Reactives = Reactive behavior management; 
Approvals = Approval and tangible reward; Disapprovals = Disapproval and punitive consequence; OTRs = Opportunities to respond. ICC= Intra-class correlation coefficient. 


